Summary

• Adds a single warmup forward pass through pipeline_module before the timing loop in run_inference_and_decode_pre_decoder_memory
• Triggers torch.compile lazy compilation so the JIT cost does not inflate the first-batch timing measurement
• Guard: only runs when trt_context is None and _applied_compile (torch-only path with compile enabled)
• CUDA sync after the warmup pass on GPU devices
• Warmup logic extracted into _maybe_warmup_compile helper with 5 unit tests

Motivation

Without this, the first batch in the timing loop bears the full torch.compile lazy-compilation cost, skewing Phase Timing numbers — especially at low sample counts (PREDECODER_INFERENCE_NUM_SAMPLES=1):

With large sample counts the JIT cost gets amortised naturally, but at small counts it dominates and makes Phase Timing numbers misleading. Proposed by Igor Almeida Baratta; approved by Ben Howe.

Test plan

• Existing unit tests pass (test_inference_latency_timing.py, test_tensorrt_fallback.py)
• Run with PREDECODER_INFERENCE_NUM_SAMPLES=1, confirm first-batch model-forward time matches steady-state
• Run with TRT enabled, confirm warmup block is skipped
• CI green

🤖 Generated with Claude Code